Skip to content

Reactions in an encrypted Tiny conversation go out sealed and fold onto the message - #259

Closed
unRekable wants to merge 8 commits into
mainfrom
fix/tiny-e2ee-reaction
Closed

unRekable wants to merge 8 commits into
mainfrom
fix/tiny-e2ee-reaction

Conversation

@unRekable

@unRekable unRekable commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

What changed and why

A reaction in an encrypted Tiny conversation was refused: the sealed send path asked the plaintext reactions route (PUT/DELETE /v1/messages/{id}/reactions), which the server answers with 400 e2ee_required for any encrypted message, and the client never called the server's encrypted route (POST/GET /v1/e2ee/conversations/{id}/reactions). The receive side had no arm for the server's e2ee.reaction event either, so nothing could have merged even if it had gone out. Fixes #234.

The sealed design. A reaction is now a ciphertext of its own, opaque to the server: the envelope gains Kind::Reaction carrying text = emoji, reply_to = target message id and a new on word (never serialized for messages or edits, so their wire bytes are unchanged). The ciphertext rides in the group the conversation serves now while its authenticated data names the conversation it was posted to (encrypt_in/decrypt_in) — after a repair the two ids differ, and the server checks both words at once, so a reaction can never be replayed onto another conversation that shares the group. Each device folds every blob it can open, in the server's list order, into who-is-on-each-(message, emoji); taking a reaction back posts another blob with on: false (the encrypted route has no delete). Counts are the device's fold, not a server word.

The fold is durable (review round two). Reaction chips live in the store's message rows, like every plaintext network's, written as blobs fold. The fold's bookkeeping — cursor, seen blobs, owed pairs, the on-map — persists as a sealed reactionfold row in the conversation's sealed cache (the same sealed append-only file that keeps own-post rows), so a restart continues from the device's cursor instead of re-opening historical blobs, whose keys forward secrecy has already spent. Two consequences, both covered by tests: a fetched page cannot wipe stored chips (a sealed wire row never carries reactions; Store::ingest keeps what a sealed row's store body folded), and a blob that cannot open yet holds the cursor — retried each fold, never skipped — until it opens or a bounded per-process budget gives up on it. Pairs whose message was not stored yet are kept owed and applied when the message lands or the next page does; pairs are validated at fold time and unclaimed ones are dropped after a bounded number of keeps.

Review findings (round one) and their resolutions

Finding Resolution
C1 — client_id over the server's 64-char cap r-<uuid> (38 chars); both fakes enforce the real rule (empty / >64 / control chars → 400 invalid), and the repair fake answers uuid-shaped ids
I1 — rebuild could resurrect a taken-back reaction kept-rows pre-pass removed; the list folds every blob in server seq order (own blobs from the cache), swept blobs last in posting order (KeptReaction gained the blob's seq); restart assertion in the regression test
I2 — blobs marked seen before their effect reached the store owed pairs kept per conversation and applied when the message lands or the next page is fetched (device cannot know what the store holds, so the ack set is exact and bounded)
I3 — transient open failure dropped a blob permanently only NoGroup/NotAMember-class marks seen; everything else retries — and since review round two, holds the cursor while it does (Blocking 3)
Major — sealed react arm discarded an unapplied pair the arm settles the pair owed on the same held lock before answering Ok (reply)
M1 live events trigger the list fold (strictly seq-correct); the frame is only the trigger
M2 decode mirrors encode: no target, no on, or an attachment along is not a reaction
M3 tracing::debug! on every retryable open failure, same shape as the message path
M4 a 404 says "This server is too old to take encrypted reactions." instead of the unreachable fallthrough
M5 fold.seen is a BTreeSet
M6 docs sentence: plaintext-era chips are replaced by the fold

Review round two (durable fold) — findings and tests

Finding Resolution
Blocking 1 — reopening wiped the chips Store::ingest keeps stored reactions for Seal::Encrypted rows whose wire carries none (a sealed wire row never can); chosen over a re-apply hook because it has no coupling to fold state at ingest time and keeps Stored::Same dedup. Test: a_conversation_reopened_keeps_its_sealed_reaction_chips
Blocking 2 — restart re-decrypted history (impossible; key storm) the fold persists in the sealed cache; restart seeds from it, the cursor continues, chips live in store rows. Test: a_restarted_device_still_sees_the_reaction_counts — restart, one catch-up ask, counts intact, and a new reaction folds immediately (a lost cursor would stall behind the spent old blob)
Blocking 3 — Retry blobs were skipped the cursor advances row by row and stops at a Retry blob; a bounded budget (MAX_REACTION_RETRIES = 3, per process) gives up so it cannot hold the cursor forever; a restart grants a fresh budget. Test: a_reaction_that_cannot_open_yet_is_retried_when_its_epoch_arrives — a third person's join moves the group to epoch 2; the blob folds only after Bea catches up, and the fake's log shows it asked for twice
Major — sealed react arm discarded an unapplied pair settled owed on the same lock before the Ok reply, like echo and page

Accepted pushback on the two-device repair ceremony

The generation-1 flow test runs Device::react after submit_repair and the fake checks the real MLS words on the reaction ciphertext (group = the binding's group, AAD names the conversation), plus an engine-level split proof (crates/e2ee/tests/reaction.rs) and a receiver-side AadMismatch proof with a real receiver (groups.rs). A full two-device repair ceremony in e2ee/fake.rs was not built: after submit_repair, the second device must adopt the canonical binding from the /generations listing rather than submit its own (two client-constructed bindings for one generation disagree, and the client refuses a mismatched receipt) — faking that faithfully would mean reverse-engineering the engine's adoption flow and testing assumptions instead of the product. The coverage above is equivalent.

Type

  • feature
  • fix
  • perf
  • refactor
  • chore
  • docs

Platforms

  • windows
  • macos
  • linux
  • android
  • ios

The change is desktop chatd only (the phone proof build does not run desktop chatd; the scenario says why it is not applicable there).

Proof

Scenario: e2e/scenarios/tiny-e2ee-reaction

Lab run: attempted from this checkout; blocked on the lab actor token (see the Lab paragraph below).

Compatibility

  • Envelope: the only protocol change is an optional on field inside the encrypted plaintext, written for reactions only; older decoders ignore unknown fields, and messages/edits are byte-identical to before.
  • Older server: the encrypted reactions route answers 404; the client now says "This server is too old to take encrypted reactions." instead of claiming the network is unreachable. Plaintext reactions stay refused by that server, as before.
  • Older client: it ignores e2ee.reaction events and never posts reactions itself, so nothing changes for it; the server still refuses its plaintext reactions, so no downgrade path exists.

Risk and rollback

The riskiest surface is the fold and its persistence. A wrong fold shows a wrong chip; the fold is rebuilt from the server's blob list at any time and the chips live in the store, so a revert of this pull request restores the previous behaviour (reactions refused in encrypted conversations) without data damage. No setting, no server deploy, squash-revert is enough.

Lab

Attempted python e2e/lab/lab.py pr 259 --platforms windows,linux from this checkout; blocked the same way as #257 — the lab needs its actor's token (gh auth token --user itz4blitz) and this machine's gh is signed in as a different account, so lab.py fails before it reaches any host:

subprocess.CalledProcessError: Command '['gh', 'auth', 'token', '--user', 'itz4blitz']' returned non-zero exit status 1.

No e2e-lab/* or e2e-proof status could be posted from this side. Local evidence already recorded:

  • Base fails by timeout, by construction: crates/chatd/src/main.rs at origin/main has no tiny-e2ee-reaction arm (0 mentions), so a base build ignores TINYCLIENT_PROVE=tiny-e2ee-reaction, keeps serving, and the driver's 90 s wait expires — exit 1. The same mechanism the keycap-reaction and first-encrypted-message drivers document.
  • Branch passes: the driver runs this branch's real chatd binary against the in-process MLS fake (which refuses plaintext reactions with 400 e2ee_required exactly as the real server does) and exits 0 — [ok] a sealed reaction lands and echoes back, prove-exit:0, steps.jsonl written.
  • The unit red fails on the base commit with the exact reported refusal (That didn't work. Check it and try again.) and passes on this branch.

Release note

Reactions work in encrypted Tiny conversations: they go out sealed, survive a restart and a reopened conversation, and taking one back works too.

…t route the server refuses, and the test expects it sealed with an echo that lands (red)
…folds rebuilt in the server's order, owed reactions applied when their message lands, and a reaction that cannot open yet is folded again
@unRekable
unRekable requested a review from itz4blitz as a code owner November 14, 2026 19:23
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Tiny Code Review

⚠️ Code review unavailable · run

the AI provider rate-limited the request

This is advice, not a gate. Re-run with the agent:review label or a /review comment.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tiny Code Review

🟡 1 finding (0 blocking).

The sealed-reaction design is coherent: a new Kind::Reaction envelope carried over the server's encrypted reactions route, AAD bound to the conversation it was posted to, a per-device fold for counts, and kept own-post rows so a device's own echo lands without MLS. Wire compatibility is preserved (on is Option with default and skipped on the wire; message/edit bytes unchanged), the plaintext route is gone with no fallback, and tests cover the envelope, the AAD binding, the repair case, and the end-to-end fold. I found one real gap: a reaction sent to a message this device has not yet stored is silently dropped from the UI until a later page fetch, because the apply failure is not kept owed the way the echo path does.

Residual risk

  • Each e2ee.reaction socket event spawns a fresh thread that takes the device mutex and does network list fetches; a server flooding events could spawn many threads and stall other device users. Existing handlers follow the same shape, so this is not new to this diff's pattern.
  • If the server's POST answer omits id, react keys the kept row under an empty blob id (reaction.), so the device's own echo would not fold; real servers return an id.
  • apply_reaction rejects non-emoji landed.emoji by returning false, which keeps the pair pending forever rather than dropping it; pending is bounded by (message, emoji) pairs.
  • Fold counts are per-device; a conversation that previously had plaintext reactions will show those chips replaced by the fold, as the docs now state.

Agent run

match result {
Ok((device, landed)) => {
drop(device);
if let Some(landed) = landed {

This comment was marked as outdated.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed as suggested: the sealed react arm now settles the pair owed on the same held lock when apply_reaction reports the message is not stored (messages.rs, the Ok((device, landed)) arm — Device::settle_reactions(conversation_id, &[], &owed)) before the Ok reply. The later echo still adds nothing new, but the owed pair is applied when the message lands (the message.created arm drains it via owe_reactions) or the next page does.

@itz4blitz itz4blitz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. The security side reads sound, but sealed reactions do not persist. Everything below is from reading the code path end to end at the PR head; nothing was built or run, and line numbers are approximate.

Blocking

  1. Reopening a conversation wipes the chips. fetch_page in messages.rs (~1426-1445) calls ctx.history with rows mapped from the wire, which carry no reactions for sealed messages. Store::ingest (store.rs ~2218-2234) replaces the stored body, and reaction_changes only returns pairs changed in that pass, so nothing is re-applied. Backfill runs on every newest-page open and after handshakes, so this is the normal path.

  2. Other people's reactions do not survive a restart. Folds are memory-only (e2ee.rs ~1078) and only our own blobs are kept. On restart fold_list re-decrypts every historical blob from after=0, but each key is already spent (messaging.rs ~319-329 returns Undecryptable). If Ada and Bea both reacted, Ada's count goes back to 1 after she restarts, and with (1) Bea's chip is gone for good. It also adds up to 800 failed decrypts per launch, each with a checkpoint restore and group reload under the device lock, on top of the SecretReuseError lines already in the logs.

  3. "Retry" never retries. fold_list sets fold.at = newest (~2926) even when fold_one returned Retry (~2970). A reaction that arrives before its commit is processed is skipped until restart, which contradicts the I3 claim in the description.

Non-blocking

  • Pending pairs never expire: a member can post invalid emoji or unknown message ids, and owe_reactions retries them on the socket thread under the device lock on every message.
  • The swept-kept pass (~2877-2892) runs after a list capped at 800, iterates in uuid order rather than time order, and applies one row per pair per call, so an add and a take-back can flip. Plausible, not confirmed.
  • If the server's answer lacks id, the fold marks "" as seen and nothing is persisted.

Checked and sound

  • The reactor is taken from the MLS credential, not the payload, so A cannot react as B.
  • The authenticated data binds kind and conversation.
  • There is no plaintext fallback: the sealed arm returns and react_generation is gone.
  • The decoder is strict and size-capped.
  • Old clients drop the unknown event and an old server's 404 is handled.

Tests

The tests do exercise the sealed route and would fail on main, but none covers the three blockers: the "restart" case only clears Ada's own fold, and there is no receiver restart, no refetch after a reaction, and no Retry case. Please add those three, and get the lab scenario to run (there is no e2e-proof status on this PR).

…ursor and owed pairs are kept sealed beside the cache, a blob that cannot open yet holds the cursor until it opens or its tries run out, and a restart neither refolds history nor loses a count
@unRekable

Copy link
Copy Markdown
Collaborator Author

Re: review round — all findings addressed (head 18fb2a29)

The three blockers shared the root cause you named: the fold was memory-only, and forward secrecy makes any "restart refolds from after=0" design dead (spent keys → Undecryptable storm under the lock). The fold is now durable, in two places with one rule each:

Design. Chips live in the store's message rows (Message.reactions, exactly like every plaintext network), written through apply_reaction → ctx.message → Store::ingest as blobs fold. The fold's bookkeeping — cursor at, seen, owed pairs, the on-map — persists as a sealed reactionfold row in the conversation's sealed cache (same append-only, sealed-under-the-cache-key file the kept own-post rows already use). Why the sealed cache and not the store: it is Tiny-e2ee-internal bookkeeping (reactor ids, blob ids, retry budgets) that belongs under the same sealing as opened history, it needs no cross-network schema change to the shared SQLite store, and the reserved-row machinery already existed. Restart: the engine and cache load from the device's directory, the fold seeds from cache.fold, the cursor continues at at, and the chips are already in the store rows — no blob is re-opened.

Blocking 1 — reopening wiped the chips. Store::ingest (store.rs, the before query now moved above the body build) keeps the stored reactions when the incoming row is Seal::Encrypted and carries none — a sealed wire row cannot carry reactions, so keeping them is the exact contract, in one place, and a row whose reactions were preserved compares equal (Stored::Same), so a plain reopen does not even rewrite. Chosen over "history triggers a re-apply" because re-apply needs the fold's per-pair state at exactly the moment ingest is replacing bodies; preservation has no such coupling. Regression test: a_conversation_reopened_keeps_its_sealed_reaction_chips (react → chip → two backfills → chip stays).

Blocking 2 — restart re-decrypted history. The persisted cursor starts the catch-up where the device left off; folded chips live in store rows and need no re-open. Regression test: a_restarted_device_still_sees_the_reaction_counts — Bea restarts (fresh Device over her own directory), sees count 1/not-mine from the persisted fold, catches up with exactly one list ask, and a new reaction after the restart folds immediately — the discriminator: with a lost cursor the old blob would be re-opened (impossible, key spent), stall the cursor, and the new one could not land.

Blocking 3 — fold_list advanced past Retry blobs. The cursor now moves row by row: Folded::Changed/Passed advance it, Folded::Retry stops the pass before that row's seq, so the next fold refetches from the waiting cursor and retries. A bounded per-process budget (MAX_REACTION_RETRIES = 3) gives up on a blob that never opens so it cannot hold the cursor forever; a restart grants a fresh budget, which is the intended recovery. Regression test: a_reaction_that_cannot_open_yet_is_retried_when_its_epoch_arrives — a third person's join moves the group to epoch 2; Bea folds the epoch-2 blob before catching up (nothing folded, nothing seen, cursor waits), then after sync the very same blob lands, with the fake's request log showing the blob asked for twice.

Major (messages.rs sealed react arm). Mirrored the echo handler: apply_reaction's false now routes the pair into Device::settle_reactions(conversation_id, &[], &owed) on the same held lock, before the Ok reply. No path discards an unapplied pair any more.

Non-blocking, all taken.

  • Pending expiry/tombstones: pairs are validated at fold time now — a blob whose target fails safe_id or whose emoji fails unicode_emoji is dropped and marked seen at the fold, so malformed pairs never enter pending; well-formed-but-unclaimed pairs are dropped after MAX_OWED_KEEPS = 10 settles. owed_reactions/settle_reactions(applied, failed) replace take/keep.
  • Missing id in the POST answer: the device no longer keeps a reaction. row or folds under "" — it logs a warning and skips (the blob will be retried from the list like any unopenable one).
  • Swept-kept pass ordering: KeptReaction gained the blob's seq (the POST answer carries it; #[serde(default)] keeps older caches loading) and the swept pass now folds in (seq, id) order — posting order — so an add and a take-back can no longer flip; the per-pair single apply is then correct because the last blob in order wins.

Lab: still environment-blocked here (lab actor token — gh auth token --user itz4blitz fails from this checkout, same as #257); the scenario's base-fails/branch-passes local evidence stands in the description.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tiny Code Review

🟡 2 findings (0 blocking).

The sealed-reaction design is sound: reactions ride as MLS application messages with conversation-binding AAD, the envelope's new on word is serde-defaulted and left off messages/edits so wire bytes are unchanged, and tests cover the binding, the retry and the round trip. Two real defects remain: the fold state grows without any cap and is re-appended in full on every fold, and the socket thread now takes the device mutex on every incoming message of an encrypted conversation, which the same diff itself avoids for e2ee.reaction.

Residual risk

  • The e2ee.reaction handler drops the spawned thread's JoinHandle and silently loses the fold nudge if the spawn fails; the next page fetch recovers it.
  • An empty on entry is left in FoldState.on when the last reactor of a pair takes it back, so the map slowly accumulates dead keys.
  • keep_reacted in Device calls self.cache(...) twice (two lock acquisitions) where one would do.

Agent run

Comment thread crates/chatd/src/adapters/tiny/e2ee.rs Outdated
#[derive(Clone, Default, PartialEq, Serialize, Deserialize)]
pub(crate) struct FoldState {
at: i64,
seen: BTreeSet<String>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Major

Every reaction blob id ever folded is kept in seen forever and in memory, and set_fold re-appends the whole FoldState (all seen ids, all on sets, all pending pairs) as one sealed line to cache-{conv}.log on every fold. A busy conversation's cache file grows roughly quadratically in the number of reactions ever posted and is never compacted, and the in-memory seen set grows without bound for as long as chatd runs — a disk- and memory-growth hazard driven by how much the network sends.

Why this diff causes it: FoldState has at, seen: BTreeSet<String>, pending, on with no size cap; set_fold calls write_sealed(key, "reactionfold", state) which appends the full serialized state each time; apply and the fold loop only ever insert into seen, nothing prunes it.

Smallest fix: Cap the persisted state: keep seen only for ids at or above at minus a window (or drop ids older than the server's list can still return, since the after cursor already prevents refetching), and prune empty on/pending entries in apply/settle. Add a test that folding N blobs keeps the persisted line bounded.

} else {
deliver(ctx, saved, names, &conversation, &wire);
}
owe_reactions(ctx, saved, &conversation);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Major

This runs inline in on_live's message.created branch, i.e. on the socket's thread, and takes the device's std::sync::Mutex. That mutex is held across network I/O during syncs and repairs (operation_deadline exists for exactly that), so a repair in flight can stall the receive socket for every incoming message of an encrypted conversation. The same diff explicitly moves the identical work off the socket thread in the e2ee.reaction branch ('Off the socket's thread: the event only says the server's list moved'), so the two paths disagree.

Why this diff causes it: The e2ee.reaction arm spawns a tiny-e2ee thread before calling owed_reactions/settle_reactions, while owe_reactions locks device.lock() directly on the live path, and fetch_page also folds while holding the lock but on a worker.

Smallest fix: In the message.created branch, do what the e2ee.reaction arm does: hand conversation_id and a cloned ctx to a spawned tiny-e2ee thread (or a single fold worker) that locks the device and applies the owed reactions, instead of locking on the socket thread.

itz4blitz added a commit that referenced this pull request Nov 17, 2026
@itz4blitz itz4blitz mentioned this pull request Nov 17, 2026
3 tasks
unRekable and others added 2 commits November 18, 2026 12:09
…ad of appended whole and keeps only what the cursor can still deliver, a blob that cannot open three folds is skipped for this session and the first fold after a restart steps back below it, the reaction list is paged without the device mutex held and a page applies only while the cursor still stands where it was asked for, rows without an id move the cursor so a page of them is never asked for twice, and a taken-back reaction leaves no empty entry behind
@itz4blitz

Copy link
Copy Markdown
Collaborator

Addressed the last remaining review finding in \7ef2a8f4:

  • Offloaded \owe_reactions\ in \on_live's \message.created/\message.edited\ branch onto a spawned \ iny-e2ee-owed\ thread (matching \e2ee.reaction) so the receive socket thread never blocks on the device mutex during syncs/repairs.
  • Removed 3 stray _pycache_/*.pyc\ files that were committed to the branch.

All blocking findings from the earlier review round (\18fb2a29, \

@itz4blitz

Copy link
Copy Markdown
Collaborator

Superseded by and merged in #903 (d387971 — Release 2026-11-20 combined PR).

@itz4blitz itz4blitz closed this Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reactions in an encrypted Tiny conversation are refused by the server

2 participants